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ABSTRACT 

This document looks at the research that has been 
conducted on the student development concept and considers its 
nature, effectiveness, and deficiencies. Research in student 
development is defined as research that attempts to ascertain if a 
particular theory-based student development intervention or treatment 
resulted in the theorized effect or result. Also included is research 
that attempts to test or validate student development theory and 
research that deals with peripheral student development issues such 
as instrument design and validation; descriptions of conditions or 
programs; a:»d philosophical ruminations about student development. A 
review of the research literature on the efficacy of student 
development interventions is summarized and its conclusions are 
discussed: that the amount of formal research on student development 
over a 14-year period (ending in 1987) was disappointing; that 
studies generally lacked control or comparison groups; and that 
evidence appeared to endorse the probability of a student development 
effect. The next section of the paper looks at research conducted 
since 1987. This more recent research conducted from 1987 through 
1990 is then compared to research conducted from 1977 through 1980 
and a set of 11 observations and recommendations is offered. (NB) 
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My assignment this afternoon is to look at the research that has been 
done on the student development concept and to come to some sort of a judgment 



^§ concerning its nature, effectiveness, and deficiencies. I'll conclude with 

q several observations and recommendations. 

What do I mean when I talk about research in student development? What 
I have in mind is research that attempts to ascertain if a particular theory- 
based student development intervention or treatment resulted in the theorized 
effect or result. Did it work? I'm also interested in research that attempts 
to test or validate student development theory, as well as research that deals 
with peripheral student development issues such as instrument design and 
validation; descriptions of conditions or programs, i.e., "How many student 
development educators present at ACPA conventions?"; or even philosophical 
ruminations about student development. 

Thrasher and Bloland (1989) 
I'm first going to summarize for you the only published article of which 
I'm aware that has examined the research on the efficacy of student 
development interventions. I refer, of course, to the review of the research 
literature that Fred Thrasher and I published in the Journal of Counseling 
and Development (Thrasher & Bloland, 1989) . 

What we did was to survey the research literature for every published 
work, regardless of quality, that reported on the implementation and 
evaluation of a theory-based student development intervention program between 
1973 - about the time student development as a concept began appearing in the 
literature - to 1987. The studies reviewed had to 1) be grounded in student 
development theory, 2) be a college-level intervention and 3) have been 

o 

evaluated. The search yielded about 145 documents that appeared to meet the 
CVi criteria; most of which, however, di. lot qualify upon inspection. 

CM 
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It soon became evident that there were two types of studies: intentional 
interventions and incidental interventions, those dealing with "intentional 
interventions" involved programs that had been specifically designed to bring 
about a change in some student development variable. These studies were most 
often experimental or quasi-experimental and could be termed "proactive" . 

The second type of intervention was termei "incidental" if it was an 
accepted on-going dimension of the college experience, i.e., the effect of 
residence hall living, participation in student activities, or just plain 
college attendance. In other words, the environment was the treatment. Any 
developmental gains were seen as incidental to the experience itself. These 
studies were most likely to be ex post facto or correlational investigations. 

Of the 12 studies categorized under the "intentional" rubric, we were 
able to identify 4 developmentally designed program interventions of which 2 
yielded significant results. There were 6 developmentally designed academic 
courses of which 4 presented significant positive results, 1 reported mixed 
results, and 1 was non-significant. There were 2 comprehensive student 
programs, both of which showed significant results, 1 positive and 1 negative. 

Although 8 of the 12 studies yielded what might be termed significant 
and positive results, there were some serious flaws in most of them. For 
example, significance was obtained for only a portion of the variables 
studied; one study relied on positive student comments; the reliability and 
validity of some of the instruments were questionable or not reported; several 
studies did not employ control groups leaving a question concerning whether 
the intervention was, in fact, the major factor in the significant differences 
observed; most designs were not tightly drawn and extraneous variables were 
present; and the external validity or generalizability of several studies was 
seriously compromised. 

Moving on to the "incidental" interventions, we found 15 studies that 
met our criteria. The 4 studies on residence hall living had positive 
results, 3 significant. Two investigations on student activities were noted, 
1 positive and significant and 1 with inconclusive results. There were 4 
studies looking at the total college experience itself as the developmental 
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intervention; 3 were positive of which 2 were significant while the 4th 
yielded variable results. There were another 5 studies whose interventions 
didn't fit the other classifications. One of these was not significant while 
4 were significant, 3 positive and 1 negative. 

Most of the investigations of incidental interventions suffered from the 
differential selection of subjects - perhaps the students comprising the 
comparison groups, i.e., residence halls students vs. commuter students - were 
different prior to the incidental intervention; the external validity of many 
studies was questionable; self -reported data were often used without 
acknowledgment . 

We concluded on the basis of our extensive review, utilizing somewhat 
restrictive but rigorous criteria, that 1) the amount of formal research on 
student development over a 14-year period was disappointing; 2) these 
studies generally lacked control or comparison groups, thereby lending 
themselves to a number of rival hypotheses; 3) however, even given these 
problems, the preponderance of what little evidence we were able to uncover 
appeared to endorse the probability of a student development effect. 

Research Production in Student Development 
Our review (Thrasher & Bloland, 1989) ended with several 1987 studies. 
As I thought about this paper I decided to look at what has occurred since 
1987 in research on student development. My overview consisted of a content 
analysis of the Journal of College Student Development (formerly the Journal 
of College Student Personnel), the NASPA Journal, and the Journal of the 
National Association for Women Deans, Administrators, and Counselors, now 
called Initiatives. After I had completed this analysis of some 598 articles 
I thought it would be interesting to compare these recent figures with a 
similar four-year period, 1977-1980, a period shortly after the beginnings of 
the student development movement but sufficiently removed so that a literature 
could be expected to have grown. This earlier period added another 498 
articles for a total of 1184 articles examined. 

I was searching for articles in the three journals that met several 
broad criteria: 1) they described themselves in the title or the abstract as 
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somehow student development related; that is, they tested student development 
theory, dealt with research methodology relevant to student development, 
examined student development assessment instruments, or reviewed related 
literature. If student development was not mentioned in the title or abstract 
I eliminated it even though it may have been developmental ly oriented. 2) 
They had to be research reports, that is, the study attempted to investigate, 
mostly by empirical means, some facet of student development. I was not 
interested in research on any other aspect of student affairs - just student 
development research. Using these criteria, I eliminated 76 essays on student 
development from my analysis. 

What I found was most interesting - and disappointing. I'm first going 
to throw some figures at you. They won't be difficult to handle because, 
unfortunately, we are not talking about a very large N. And please bear in 
mind that my classification of a particular study might very well differ from 
yours . 

The number of studies devoted to research in these three journals 
doubled in the period between 1977-80 and 1987-90. Most of the increase was 
represented by the Journal of College Student Development which almost tripled 
its student development research content, from 12 to 33, although the ratio to 
its total content still was not impressive, from 3.5% in the earlier period to 
13% in the later period. The NASPA Journal dropped its student development 
research content from 4% to 2% while the NAWDAC Journal published only 2 
research articles on student development. 

An examination of the types of research designs utilized provide a rough 
index: of the sophistication of the research being published. The most 
powerful approach for determining the effect of one factor, or factors, upon 
another would be the true experimental design. You may recall that the true 
experimental design is characterized by the random assignment of subjects to 
treatment and control groups so that the two groups have no major differences 
between them before treatment begins. The true experimental and the quasi- 
experimental designs (no random assignment, intact groups) permit the 
researcher to make more definitive statements on cause and effect, which ought 
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to be a prime consideration in assessing the effect of a student development 
intervention . 

Having said that, I could identify only 4 studies in all three journals 
in the two four-year periods surveyed that could be readily categorized as 
either quasi or true experimental designs . In fairness it should be noted 
that it is most difficult to achieve random assignment in field studies 
although quasi -experimental studies which can use pre-formed, intact groups 
ought to be feasible. 

Of the non- experimental designs those describing phenomena involving 
relationships between variables without suggesting cause and effect, I found 5 
descriptive studies, 7 correlational studies, and 8 survey investigations. 

I defined ex post facto , or causal -comparative, designs as those that 
study cause and effect by comparing groups of students on variables that 
cannot be manipulated experimentally because the causal events have already 
occurred. For example, an ex post facto study might ask "Are there 
significant developmental differences between inner-city students and those 
who came to college from a rural background?" There were 12 such studies, 8 of 
which were theory-based. It is notable that 11 of the 12 were conducted in 
the 1987-90 time frame; I could identify only 1 in the earlier period. The ex 
post facto design appeared to be the most popular design and lends itself well 
to the study of student development effects because it does not require 
experimental manipulation while yet accomodating itself to robust statistical 
treatment . 

A fourth major design category lumps all qualitative methodologies 
together. In spite of an increasing interest in promoting more qualitative 
designs I was able to identify only 6, 4 of them in the recent 4-year period, 
and 2 in the earlier period. 

Longitudinal studies have also been called for in order to measure the 
long-term effect of student development interventions. Each time I ran into a 
longitudinal study I classified it as such although several could just as well 
have been categorized as correlational, ex post facto, or quasi-experimental. 
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There were 8 studies that could reasonably be classified as longitudinal, 6 of 
them conducted and reported in the later time frame, 1987-90. 

I established several other categories for studies that didn't fit the 
generally recognized research design divisions but which reported on studies 
that contributed to the on-going research needs of those studying student 
development. For example, there were 3 articles dealing with research 
methodologies that may be useful to student development investigations. Five 
articles discussed the development of research instruments concerned with the 
measurement of developmental variables, and there were 2 reviews of the 
research literature. 

Some Observations and Recommendations 
After completing these two analyses of the research literature on 
student development, I was left with a number of impressions. What follows is 
a series of eleven observations that either imply certain recommendations or 
make them explicit. 

1. As noted earlier, there was a surprising number of studies reporting 
significant results that did not employ control or comparison groups. As a 
consequence, any observed effect could lend itself to rival hypotheses - that 
is to say, it may not have been the student development intervention being 
studied that caused the significant result. 

2. Because of the difficulties inherent in implementing true 
experimental conditions when one is doing research with students in their 
environment, it is understandable that so few of these are done. Yet, if one 
designed a specific and circumscribed intervention on an easily measured 
variable, as in a developmental course, it ought to be possible to employ 
random assignment and control groups. The same is true of quasi -experimental 
studies, or longitudinal field studies. 

3. A major problem is insufficient time in treatment. Can one 
reasonably expect that a developmental variable would be affected by a 
treatment of, say, one hour a week for six weeks, for example? The pressure 
for results, a dissertation or publication, means that the quick and dirty 
approach is almost endemic in the field. One can't blame the researchers but 
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the result is less than potent treatments. Consider also that only the most 
statistically significant studies hit the journals. How many unpublished 
dissertations found no student development effect? 

4. Because practically no experimental research is being done, the 
results are subject to selection bias, i.e., the posttesting differences may 
have been inherent in the sample selected and not due to the treatment. 

5. Certainly differential treatment has been a problem in several 
studies - groups in different institutions or using different leaders or 
instructors for comparison groups. Any significant developmental change may 
well be due to inherent differences in the institutions or leaders, not the 
treatment . 

6. There is practically no replication of studies; in part, I suspect, 
because the published studies don't provide enough information about the 
nature of the treatment for others to duplicate it. How do we know that the 
observed effect is not simply idiosyncratic to that situation unless we can 
see the same effect in different settings and with different populations? 

7. Methods of data analysis have become much more sophisticated today 
than they were ten years ago. We have such methodologies as discriminant 
analysis, LISREL, canonical correlation, path analysis, causal modeling, 
factor analysis, MANOVA designs, and the like being utilized. 

8. Researchers today are using a wider variety of student development 
theories and models than was true ten years ago, probably because more are 
available 

9. Very little qualitative investigation on student development is 
published in our journals; this despite the open invitation from Brown in his 
first editorial as Editor of the .Journal of College Student Personnel (1983) 
in which he said that "solid experimental research must be balanced with good 
qualitative studies" (p. 3) . He later stated that the so-called emerging 
paradigm may best be investigated through qualitative approaches that "often 
let the theory emerge from the data rather than be preordained before 
conducting ... studies" (Brown, 1988, p. 99). 
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10. There has been some discussion of meta-analysis (Manke & Erwin, 
1988, pp. 549-552) or integrative research reviews in which the results of a 
number of related studies on a single topic can be pooled. Unfortunately, in 
the area of student development, there haven't been enough studies on a single 
topic to integrate - thus the need for replications. In fact, there are very 
few reviews of the research literature at all. I was able to identify only 2 
in the 8 -year period I sampled. It is imperative that, as the body of 
research on student development expands, we begin to synthesize it, whether 
through traditional literature reviews or meta-analysis. 

11. And finally, there is the variable problem. We not only need 
better and more sensitive instruments but need to reduce the current anarchy 
in our studies of student development. Each lonely graduate student or 
isolated professor picks out his or her favorite variable and runs a study 
which then reigns as the only study ever conducted or likely to be conducted 
on that variable, hardly the way to study a unified coherent field. Without 
restricting in any way the creativity of independent researchers as they 
follow their intuitions, we might collectively specify a set of variables that 
are important to generalized student development, devise a set of standardized 
instruments to measure them, and then concentrate a number of studies on these 
variables in many institutions. 

It would then be possible for a body of unified knowledge to emerge from 
the collective data and we could begin to see if student development 
intervention and programming really works. 

Summary 

In summary, I will readily admit that my criticisms of the extant 
research on student development might well be made of many other applied 
fields, particularly in the human sciences. Great credit is due the person who 
undertakes to thread his or her way through the mine fields of research in an 
effort to shed light on a research problem of concern to the field and its 
advancement . 

But, at the same time, good intentions are not enough in the grim hard 
marketplace of ideas. Unless our research can begin to come up with some 
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well -designed studies that yield positive data with the fewest threats to 
internal and external validity and on variables that are important - that 
matter - the student development concept is going nowhere because we won't 
have anything but good intentions to back it up. And politically on the 
campus, even good research on student development may be irrelevant if the 
concept is not accepted as a worthwhile investment for an institution. 

For a field so devoted to student development that we have changed the 
name of our professional association and our journal to demonstrate that 
devotion, we have precious little research evidence to justify that wholesale 
affection - and what we do have is not very reassuring considering the need. 

We are left with the question, "Is there such a thing as intentional 
student development?". I would answer by saying, "I really don't know. There 
is some tentative evidence that developmental change may be demonstrated for 
some narrowly- focused developmental variables under certain conditions with 
specific populations - at least once! 
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